zlib: add ZIP archive support to zlib (ZipFile,ZipBuffer,ZipEntry) by pipobscure · Pull Request #64339 · nodejs/node

pipobscure · 2026-07-07T14:54:34Z

Summary

Add ZIP archive support to the node:zlib module through three classes
and a set of helpers:

ZipEntry: a single archive member, with buffered reads (content()),
bounded-memory streaming reads (contentIterator()), and
create()/createStream() for building members.
ZipFile: random access to an archive backed by a file descriptor,
reading members lazily without retaining their content and writing
new members in place; opened with open()/openSync().
ZipBuffer: a zero-copy, in-memory view over an archive already held
in a Buffer.

createZipArchive() serializes a sequence of entries into an archive
byte stream, and setMaxZipContentSize() bounds the default in-memory
decompression size. Every operation has both an asynchronous and a
synchronous form.

P.S.: my CLA should be on file and I wrote this myself so COO is declared

codecov · 2026-07-07T16:32:30Z

Codecov Report

❌ Patch coverage is 98.24856% with 73 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.31%. Comparing base (396aad0) to head (fb49aca).
⚠️ Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/internal/zip/file.js	96.41%	27 Missing and 1 partial ⚠️
lib/internal/zip/entry.js	98.44%	13 Missing and 2 partials ⚠️
lib/internal/zip/headers.js	97.26%	14 Missing ⚠️
lib/internal/zip/fs-util.js	92.00%	6 Missing and 6 partials ⚠️
lib/internal/zip/archive.js	99.37%	2 Missing ⚠️
lib/internal/zip/compression.js	99.33%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #64339      +/-   ##
==========================================
+ Coverage   90.17%   90.31%   +0.13%     
==========================================
  Files         746      760      +14     
  Lines      242776   246928    +4152     
  Branches    45741    46578     +837     
==========================================
+ Hits       218929   223003    +4074     
- Misses      15347    15411      +64     
- Partials     8500     8514      +14

Files with missing lines	Coverage Δ
lib/internal/errors.js	`97.91% <100.00%> (+<0.01%)`	⬆️
lib/internal/zip.js	`100.00% <100.00%> (ø)`
lib/internal/zip/binary.js	`100.00% <100.00%> (ø)`
lib/internal/zip/buffer.js	`100.00% <100.00%> (ø)`
lib/internal/zip/constants.js	`100.00% <100.00%> (ø)`
lib/internal/zip/content-size.js	`100.00% <100.00%> (ø)`
lib/internal/zip/dos.js	`100.00% <100.00%> (ø)`
lib/internal/zip/extra-fields.js	`100.00% <100.00%> (ø)`
lib/internal/zip/header-builders.js	`100.00% <100.00%> (ø)`
lib/zlib.js	`98.20% <100.00%> (+0.12%)`	⬆️
... and 6 more

... and 40 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Codecov flagged low patch coverage on lib/internal/zip.js and lib/internal/vfs/providers/archive.js in nodejs#64339. Add tests exercising Zip64 extra-field parsing, DOS date/time edge cases, streaming-entry state guards, decodeMemberStream/decodeMemberSync's duplicated error branches, the ZipBuffer/ZipFile iteration protocols, several on-disk ZipFile error paths, and ArchiveFileHandle's direct read/write/stat/ truncate surface plus a handful of provider-level error branches the existing tests didn't reach.

bakkot · 2026-07-07T23:15:44Z

See also #45651

pipobscure · 2026-07-08T05:42:44Z

See also #45651

Thanks @bakkot !!!

I think the time has come for it on the one hand, and on the other I added some „motivation“ links earlier. Here some more detail:

Based on this, we can modify the loader to directly load from an archive. If we do that, we get application bundles. pipobscure#5 & pipobscure#6

Which can then in turn be used to easily create application bundles: https://github.com/pipobscure/experimental-sea

So the world had changed enough that it‘s worth proposing again.

mcollina · 2026-07-08T13:00:36Z

I like this a lot. It's likely better to split this into 2 PRs, one for Zip support and one for VFS-Zip, so the Zip support could theoretically be backportable on its own.

pipobscure · 2026-07-08T16:32:01Z

As per @mcollina I split this into two PRs.

I have the vfs-provider ready to go as follow up one (as it depends on this being merged)

I also made sure that the streaming side of things was actually as clean as I intended, and added a few more tests.

bakkot · 2026-07-08T17:47:34Z

Might be worth borrowing some tests from other projects: e.g. python, go, info-zip. There's a lot of edge cases with zip files. (Python's tests are larger than this whole PR put together.)

Not all directly applicable because this doesn't include the ability to extract to disk, fortunately.

pipobscure · 2026-07-09T05:17:34Z

Might be worth borrowing some tests from other projects

I've added a bunch more tests and compared to what go/python/info-zip do. (info-zip is cli exercise, so it's not entirely clear what gets tested).

I also gave zip.js an once over and concluded that it was too large a file (it originated from separate files that I've had for ages combined into one).

So I split it back out so it would be easier to review. And gave that another look.

pipobscure · 2026-07-24T08:46:18Z

Rebased on latest main to resolve the conflict

pipobscure · 2026-07-24T09:46:37Z

The failing mac tests seem to be entirely unrelated

Failed tests:
out/Release/node /Users/runner/work/node/node/node/test/parallel/test-debugger-extract-function-name.mjs
out/Release/node --test-reporter=./test/common/test-error-reporter.js --test-reporter-destination=stdout /Users/runner/work/node/node/node/test/parallel/test-repl-user-error-handler.js

can someone verify my deduction please.

mcollina

lgtm

nodejs-github-bot · 2026-07-29T13:32:30Z

CI: https://ci.nodejs.org/job/node-test-pull-request/75288/

nodejs-github-bot · 2026-07-29T17:21:47Z

CI: https://ci.nodejs.org/job/node-test-pull-request/75295/

mcollina · 2026-07-30T06:52:35Z

Everything is green, CI is ok, we are missing another @nodejs/tsc approval as this a large PR.

marco-ippolito

LGTM

targos

I am reviewing. You can dismiss my request for changes if I don't answer by tomorrow.

panva

import { createGzip } from 'node:zlib'

This stable import emits ZIP experimental warning because of how our ESM facade evaluates the getters.

Add ZIP archive support to the node:zlib module through three classes and a set of helpers: - ZipEntry: a single archive member, with buffered reads (content()), bounded-memory streaming reads (contentIterator()), and create()/createStream() for building members. - ZipFile: random access to an archive backed by a file descriptor, reading members lazily without retaining their content and writing new members in place; opened with open()/openSync(). - ZipBuffer: a zero-copy, in-memory view over an archive already held in a Buffer. createZipArchive() serializes a sequence of entries into an archive byte stream, and setMaxZipContentSize() bounds the default in-memory decompression size. Every operation has both an asynchronous and a synchronous form. Signed-off-by: Philipp Dunkel <pip@pipobscure.com>

pipobscure · 2026-07-30T17:54:58Z

I rebased on main. Then I addressed the docs issue from @marco-ippolito and the import issue by @panva . I also go a security review from @mcollina and dealt with the issues that raised.

Here are the security-review issues we addressed:

Numbered findings

Finding 1 (High): After ZipFile.close(), retained entries/operations fell through to a raw fd that the OS could reuse, enabling cross-file read/write/close.
Finding 2 (High): add() awaited compression before queuing the mutation, so a concurrent close()/closeSync() could tear down the fd mid-add.
Finding 3 (High): zipFiles() re-opened files by path (and never fstat-checked type), allowing a symlink-swap TOCTOU and accepting FIFOs/devices.
Finding 4 (Medium): The open-time overlap check used a 30-byte lower bound instead of the exact local-header length, so a quoted-overlap archive passed ZipFile.open though ZipBuffer rejected it.

Additional concerns

A (in-place corruption): A failed central-directory rewrite can leave the on-disk archive corrupt and diverged from memory — documented as non-atomic (recovery preserved).
B (streaming ceiling): contentIterator()/stream() ignored the default decompression ceiling, allowing a zip bomb through the streaming API.
C (EOCD/ZIP64 mismatch): A ZIP64 record silently overrode contradictory classic-EOCD values with no consistency check, a parser-differential risk.
D (provisional streamed bytes): Streamed chunks are yielded before CRC verification runs at stream end — inherent to streaming, so documented.
E (FIFOs/devices): Special files were accepted as stream sources — resolved by the same fstat/type-check as Finding 3.

panva · 2026-07-30T18:01:51Z

I rebased on main.

So we can't easily see and track the requested diff now. Unfortunate as otherwise we could review increments instead.

stale, need to start over now.

pipobscure · 2026-07-30T18:17:56Z

I rebased on main.

So we can't easily see and track the requested diff now. Unfortunate as otherwise we could review increments instead.

Apologies. I thought the “requested modus operandi “ was to keep it to a single commit. If that’s wrong then I’ll change the approach for the future.

panva · 2026-07-30T18:22:08Z

Apologies. I thought the “requested modus operandi “ was to keep it to a single commit. If that’s wrong then I’ll change the approach for the future.

No worries. FYI the commit-queue can land fixup commits (autosquash), or we can apply a label to squash, or if the PR has multiple things that can/should be backported independently (e.g. part is semver-major, part isn't) we can also land with rebase.

contentIterator() (and ZipFile.stream()) must not apply the default getMaxZipContentSize() limit the way content()/contentSync() do. That default guards a single large buffer allocation, which the streaming path never makes: streaming is the bounded-memory way to read arbitrarily large members. Applying the ceiling there rejected legitimate reads of members larger than it - including the >4 GiB streaming case in test/pummel/test-zlib-zip-slow.js. Output stays bounded per chunk to the declared uncompressed size, and a caller that wants an explicit cap can still pass options.maxSize.

pipobscure · 2026-07-30T20:48:33Z

Loving tests right now. It caught me breaking streaming when doing the sec-fixes earlier.

nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Jul 7, 2026

pipobscure changed the title ~~zlib, vfs: add ZIP archive read/write support and an archive vfs provider~~ zlib, vfs: add ZIP archive support and an archive vfs provider Jul 7, 2026

This comment was marked as outdated.

Sign in to view

pipobscure force-pushed the ziparchives branch 3 times, most recently from 0f2b879 to 4258e94 Compare July 7, 2026 19:20

pipobscure force-pushed the ziparchives branch 2 times, most recently from 2a56a15 to 27ab6b4 Compare July 7, 2026 21:54

jasnell reviewed Jul 8, 2026

View reviewed changes

Comment thread doc/api/vfs.md Outdated

JakobJingleheimer self-requested a review July 8, 2026 08:40

pipobscure force-pushed the ziparchives branch 2 times, most recently from f157686 to 233f865 Compare July 8, 2026 11:29

mcollina reviewed Jul 8, 2026

View reviewed changes

Comment thread doc/api/zlib.md

This comment was marked as outdated.

Sign in to view

pipobscure force-pushed the ziparchives branch from 233f865 to 26fb1ab Compare July 8, 2026 16:22

pipobscure changed the title ~~zlib, vfs: add ZIP archive support and an archive vfs provider~~ zlib, vfs: add ZIP archive support to zlib Jul 8, 2026

mcollina reviewed Jul 8, 2026

View reviewed changes

Comment thread test/parallel/test-zlib-zip-vfs.js Outdated

pipobscure force-pushed the ziparchives branch from 26fb1ab to 701dffd Compare July 8, 2026 16:44

pipobscure changed the title ~~zlib, vfs: add ZIP archive support to zlib~~ zlib: add ZIP archive support to zlib (ZipFile,ZipBuffer,ZipEntry) Jul 8, 2026

pipobscure force-pushed the ziparchives branch from 701dffd to 03266e9 Compare July 9, 2026 05:10

trivikr added the zlib Issues and PRs related to the zlib subsystem. label Jul 9, 2026

trivikr requested a review from jasnell July 21, 2026 05:40

pipobscure force-pushed the ziparchives branch from d1afb16 to 4fdc3fe Compare July 24, 2026 08:45

mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Jul 24, 2026

This comment was marked as outdated.

Sign in to view

mcollina approved these changes Jul 29, 2026

View reviewed changes

mcollina added request-ci Add this label to start a Jenkins CI on a PR. and removed request-ci Add this label to start a Jenkins CI on a PR. labels Jul 29, 2026

github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 29, 2026

This comment was marked as outdated.

Sign in to view

jasnell approved these changes Jul 30, 2026

View reviewed changes

marco-ippolito reviewed Jul 30, 2026

View reviewed changes

Comment thread doc/api/zlib.md Outdated

panva approved these changes Jul 30, 2026

View reviewed changes

marco-ippolito approved these changes Jul 30, 2026

View reviewed changes

targos requested changes Jul 30, 2026

View reviewed changes

panva added the experimental Issues and PRs related to experimental features. label Jul 30, 2026

panva previously requested changes Jul 30, 2026

View reviewed changes

pipobscure force-pushed the ziparchives branch from 4fdc3fe to a6db6c9 Compare July 30, 2026 17:52

Uh oh!

Uh oh!

Conversation

pipobscure commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

codecov Bot commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

This comment was marked as outdated.

bakkot commented Jul 7, 2026

Uh oh!

Uh oh!

pipobscure commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mcollina commented Jul 8, 2026

Uh oh!

Uh oh!

This comment was marked as outdated.

pipobscure commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

bakkot commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pipobscure commented Jul 9, 2026

Uh oh!

pipobscure commented Jul 24, 2026

Uh oh!

pipobscure commented Jul 24, 2026

Uh oh!

This comment was marked as outdated.

mcollina left a comment

Choose a reason for hiding this comment

Uh oh!

nodejs-github-bot commented Jul 29, 2026

Uh oh!

This comment was marked as outdated.

nodejs-github-bot commented Jul 29, 2026

Uh oh!

mcollina commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

marco-ippolito left a comment

Choose a reason for hiding this comment

Uh oh!

targos left a comment

Choose a reason for hiding this comment

Uh oh!

panva left a comment

Choose a reason for hiding this comment

Uh oh!

pipobscure commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

panva commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pipobscure commented Jul 30, 2026

Uh oh!

panva commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pipobscure commented Jul 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

12 participants

pipobscure commented Jul 7, 2026 •

edited

Loading

codecov Bot commented Jul 7, 2026 •

edited

Loading

pipobscure commented Jul 8, 2026 •

edited

Loading

pipobscure commented Jul 8, 2026 •

edited

Loading

bakkot commented Jul 8, 2026 •

edited

Loading

mcollina commented Jul 30, 2026 •

edited

Loading

pipobscure commented Jul 30, 2026 •

edited

Loading

panva commented Jul 30, 2026 •

edited

Loading

panva commented Jul 30, 2026 •

edited

Loading